EEG control of devices using sensory evoked potentials

ABSTRACT

An EEG control of devices using Sensory Evoked Potentials (SEPs) (e.g., visually-evoked potentials), is disclosed. In some embodiments, a system receives a plurality of EEG signal samples; generates a stimulus locked average signal using the plurality of EEG signal samples; and determines whether the plurality of EEG signal samples are evoked in response to a pattern of stimulus.

BACKGROUND OF THE INVENTION

EEG detection systems exist that include bio-signal sensors (e.g.,electroencephalography (EEG) sensors) that allow brain waves of a userto be measured. Sensory Evoked Potentials (SEPs) are generallyinvoluntary EEG signals of a person generated when the person respondsto a stimulus (e.g., visually-evoked potentials, or EEG potentialsevoked through other senses, such as tactile-evoked or audio-evokedpotential). Thus, it is desirable to provide EEG detection systems thatcan be used for SEP applications and/or EEG control of devices usingSEPs, such as visually-evoked potentials.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments of the invention are disclosed in the followingdetailed description and the accompanying drawings.

FIG. 1 is a block diagram illustrating an EEG system for SEPs inaccordance with some embodiments.

FIG. 2 is a functional diagram illustrating an EEG control system inaccordance with some embodiments.

FIG. 3 is a functional diagram illustrating an EEG detection system inaccordance with some embodiments.

FIG. 4 illustrates an EEG detection system including an EEG sensor andreference EEG sensor mounted inside a hat in accordance with someembodiments.

FIGS. 5A-B illustrate EEG sensors in accordance with some embodiments.

FIG. 6 is another block diagram illustrating an EEG system for SEPs inaccordance with some embodiments.

FIG. 7 illustrates an EEG detection system with non-contact EEG sensorsin accordance with some embodiments.

FIGS. 8A-B illustrate LED lights for an EEG system for SEPs inaccordance with some embodiments.

FIGS. 9A-B are charts illustrating EEG data and light control signaldata for an EEG system for SEPs in accordance with some embodiments.

FIG. 10 is a power spectrum chart for sample EEG data.

FIG. 11 is a chart illustrating averaged EEG data following light onsetsfor an EEG system for SEPs in accordance with some embodiments.

FIG. 12 is a chart illustrating a correlation of light flash and raw EEGdata for an EEG system for SEPs in accordance with some embodiments.

FIG. 13 is a flow chart for an EEG system for SEPs in accordance withsome embodiments.

FIG. 14 is another flow chart for an EEG system for SEPs in accordancewith some embodiments.

FIG. 15 is a diagram illustrating different stimulus types in accordancewith some embodiments.

FIG. 16 is a diagram illustrating time domain algorithms in accordancewith some embodiments.

FIG. 17 is a chart illustrating an example of four stimulus-lockedaverages in accordance with some embodiments.

FIG. 18 is a chart illustrating an example for generating a flash-lockedaverage signal in accordance with some embodiments.

DETAILED DESCRIPTION

The invention can be implemented in numerous ways, including as aprocess; an apparatus; a system; a composition of matter; a computerprogram product embodied on a computer readable storage medium; and/or aprocessor, such as a processor configured to execute instructions storedon and/or provided by a memory coupled to the processor. In thisspecification, these implementations, or any other form that theinvention may take, may be referred to as techniques. In general, theorder of the steps of disclosed processes may be altered within thescope of the invention. Unless stated otherwise, a component such as aprocessor or a memory described as being configured to perform a taskmay be implemented as a general component that is temporarily configuredto perform the task at a given time or a specific component that ismanufactured to perform the task. As used herein, the term ‘processor’refers to one or more devices, circuits (e.g., PCBs, ASICs, and/orFPGAs), and/or processing cores configured to process data, such ascomputer program instructions.

A detailed description of one or more embodiments of the invention isprovided below along with accompanying figures that illustrate theprinciples of the invention. The invention is described in connectionwith such embodiments, but the invention is not limited to anyembodiment. The scope of the invention is limited only by the claims andthe invention encompasses numerous alternatives, modifications andequivalents. Numerous specific details are set forth in the followingdescription in order to provide a thorough understanding of theinvention. These details are provided for the purpose of example and theinvention may be practiced according to the claims without some or allof these specific details. For the purpose of clarity, technicalmaterial that is known in the technical fields related to the inventionhas not been described in detail so that the invention is notunnecessarily obscured.

The typical electroencephalography (EEG) signal that is generated from astimulus event, such as the user looking at a flashing light, is arelatively weak signal. As a result, it is not easy to detect suchsignals with the typical amount of noise (e.g., from the circuit,external sources, and/or non-relevant EEG sources) in the detectedsignal (e.g., using dry, contact sensor(s), wet, contact sensor(s), ornon-contact EEG sensor(s)). In addition, it is also not easy to detectthe signature EEG signal in a timely manner (e.g., within 2 to 3seconds). For example, systems that use lights that flash at fixedfrequencies rely on monitoring EEG signals for an increase in power atthe light frequencies. These systems and methods are generally referredto as steady-state visually-evoked potentials (SSVEPs). However, powerestimation techniques (e.g., FFT techniques) are not reliable when thelevel of noise in the EEG signal is of the same order as the signal thatis being estimated, which is often the case, especially with non-contactEEG sensors.

Techniques that rely on EEG potentials generated by thoughts orhigh-level perceptions are slow. For example, with P300 event-relatedpotentials (ERPs), the user must recognize relatively rare eventsrequiring that events be spaced relatively far apart in time (e.g., tenevents will be spaced over one minute), which limits the speed at whicha determination/action can be performed with EEG-based control.

Accordingly, a system and method that can efficiently and effectivelydetermine stimulus evoked events (e.g., SEPs, such as visually-evokedpotentials) based on EEG signals is needed.

In some embodiments, a system is provided that efficiently andeffectively identifies EEG signals associated with SEPs to control adevice. In some embodiments, a system is provided that uses flashinglights (e.g., from one or more light-emitting diodes (LEDs) and/or froma computer screen or television (TV) screen) that correspond to commandsto/from a user. In some embodiments, the flashing lights in the systemflash at distinct fixed frequencies. In some embodiments, the flashinglights in the system flash at variable frequencies in a fixed pattern orin non-periodic frequencies. The system records detected EEG signals ofthe user and determines whether/when a user is looking at one of theflashing lights. As used herein, SEPs generally refer to involuntary EEGsignals generated when the user is exposed to (e.g., rapidly) repeatingsensory stimuli (e.g., visual, such as a flashing light or anotherinvoluntary response to a visual stimulus event, audio, tactile, orother stimulus event). As used herein, SEPs do not include events basedon a user's thought and higher level perceptions (e.g., recognition of arelatively rare event, like P300s, or recognition of a grammar mistake),which generally occur after a longer period of time offset (andgenerally require a relatively slow repetition of such events so thatthe small EEG signal samples can be added and averaged foridentification purposes), and which are generally referred to asevent-related potentials (ERPs).

In some embodiments, a system is provided that uses various signalaveraging techniques on SEP signals generated in response to rapidlyrepeating sensory stimuli. In some embodiments, such as shown in FIG.18, an EEG signal from a user reacting to a rapidly repeated stimulusevent, such as the user looking at a flashing light, and a signal thatis used to control the stimulus are both measured. For example, segmentsof the EEG signal of a fixed length (e.g., 100 milliseconds (ms)) thatfollow the light onsets are first recorded. The recorded data is thenaveraged together such that the first EEG data points following stimulusonsets are averaged together, then all the second data points thatfollow stimulus onsets are averaged together, then the third data pointsthat follow stimulus onsets are averaged together, and so forth. Theresult provides a stimulus-locked average signal (or in this example, aflash-locked average signal) with an averaged waveform of the samelength as the recorded segments. For example, if the user looked at thelight, then the averaged waveform will include a characteristic shape(e.g., a positive deflection in EEG voltage about 30 ms after the lightonset time). This characteristic shape of the averaged waveform can bedetected in a variety of ways as further described herein. If the userdid not look at the light the averaged waveform will be mostly flat.

In some embodiments, the system includes more than one stimuli. Forexample, FIG. 17 is a chart illustrating an example of fourstimulus-locked averages. One of the averages has developed acharacteristic shape, whereas, the other averages are relatively flat.

The characteristic shape can be detected using various techniquesincluding comparing to a threshold at some delay from the onset,integrating the averaged EEG signal over some time period and comparingthat result to a threshold, or creating a classifier that distinguishesif the light is being attended. As another example, a prototype of anideal average (e.g., when a light is being attended) can be constructedand multiplied by the actual average. A high value result will indicatean attended light. The prototype can be constructed in a variety ofways, including computing EEG averages when a user is known to belooking at the light, or constructing an auto-regression model of EEGdata when a user is known to be looking at the light, and using thecoefficients of that auto-regression model as the data elements of theprototype.

In some embodiments, the system is used to control devices using certainEEG signals that are evoked by a user looking at flashing lights. Forexample, a control signal can also be provided to another device (e.g.,an entertainment system, an educational system, a medical system, anapplication for automobiles, and a computer executing an application)based on detected SEPs. For example, the system can include severalflashing lights that each represent a command that is used to controlthe device. When a user looks at one of the flashing lights, a uniquesignature in the EEG signal can be determined to be present in theuser's recorded EEG signals pattern using stimulus-locked averagetechniques. For example, a computing device (e.g., a programmedcomputer/laptop/netbook/portable computing device, microcontroller,ASIC, and/or FPGA) can perform an efficient and effective algorithm(e.g., classifier) that continuously checks for unique EEG signalsignatures that correspond to each flashing light. In some embodiments,the algorithm performs such determinations in real-time (e.g., computessuch determinations within about 3 seconds of the event, in this case,the flashing light(s) event(s)). In some embodiments, various parametersare adjusted to maximize the EEG signal and increase the visually-evokedpotential detection rate, such as light brightness, color, spacing,frequency, duty cycle, and the amount of visual field that is used bythe lights. When a visually-evoked potential is detected, then thecorresponding command is sent to the controlled device.

For example, the device that is controlled can be a toy, and when thesystem recognizes that one of the flashing lights is being looked at bythe user, then something fun happens with the toy (e.g., based on acommand from the system for detecting SEPs). As another example, objectswithin a video game can flash, and the game can recognize what the useris looking at (e.g., which flashing object) and incorporate that intothe game play. As another example, objects within a flight simulator ora military or other application can flash, and the game can recognizewhat the user is looking at (e.g., which flashing object) andincorporate that into the application. As another example, the devicethat is controlled can be a programmed computer, or any apparatus, thatallows a user who cannot use their hands, but needs the ability to makea system selection. As another example, the device that is controlledcan be an automobile application, in which a selection or setting for anautomobile interface for drivers and/or passengers. For example, the EEGdetection system can be in the form of a cap worn by the user and/orintegrated into a headrest of an automobile seat, and blinking/flashinglights can be integrated into a console/dashboard of the automobile forcontrolling radio, temperature, or other controls/settings, or combinedwith various other EEG applications for automobiles or other devices,such as a mental state monitor (e.g., for determining attention,anxiety, surprise, and/or drowsy states, such as for a driver of theautomobile, an airplane, or any other device).

FIG. 1 is a block diagram illustrating an EEG system for SEPs inaccordance with some embodiments. As shown, an EEG system for SEPs 100includes an EEG control system 110, an EEG detection system 130, and adevice 150. In some embodiments, the device 150 is controlled by the EEGcontrol system 110. In some embodiments, the device 150 is included withor integrated with the EEG system for SEPs, as shown, and communicateswith the device 150 using a serial or other communication channel. Insome embodiments, the device 150 is separate from the EEG system forSEPs 100 and is in communication with the EEG control system 110 using awired line or wireless communication. In some embodiments, the EEGcontrol system 110 communicates with the EEG detection system 130 usinga serial or other communication channel (e.g., wired or wireless).

In some embodiments, the EEG detection system 130 detects EEG signals ofa user, and the EEG control system 110 includes a processor configuredto perform an SEP determination algorithm (e.g., a real-timeclassification algorithm/classifier) for EEG signals detected by EEGdetection system 130. In some embodiments, various SEP determinationtechniques are used (e.g., time domain SEP determinationalgorithms/classifiers), as disclosed herein.

In some embodiments, based on the SEP determination(s), the EEG controlsystem 110 sends corresponding control signal(s) to the device 150(e.g., based on associated SEPs). In some embodiments, the EEG detectionsystem 130 sends raw EEG signal data, or in some embodiments processedEEG signal data (e.g., to filter out noise), to the EEG control system110.

FIG. 2 is a functional diagram illustrating an EEG control system inaccordance with some embodiments. As shown, the EEG control system 130includes an EEG detection communication component 112 for communicatingwith the EEG detection system 130, a processor 114 for performing an SEPdetermination algorithm for EEG signals detected by EEG detection system130, an output control 118 for communicating with the device 150, LEDcommunication 122 for communicating with one or more LEDs (e.g., aflashing LED lights system), and a data storage 124 (e.g., for storingreceived EEG signal samples and associated timing data, such as for theflashing LED lights), and a communication link 120.

In some embodiments, a programmed computer is in communication with theEEG control system 110, and the EEG control system 110 also includes anEEG data to computer component for sending detected EEG signal samplesto the computer. In this example, the computer includes a processorconfigured to perform an SEP determination algorithm for EEG signalsdetected by EEG detection system 130, and the computer can then providethe results of the analysis to the EEG control system for controllingthe device (e.g., based on associated SEPs). In some embodiments, thecomputer includes a processor configured to perform an SEP determinationalgorithm for EEG signals detected by EEG detection system 130, and thecomputer sends corresponding control signal(s) to the device based onthe results of the analysis of the EEG signal samples. In someembodiments, all or just a portion of the analysis of the EEG signalsamples is performed by the programmed computer. In some embodiments,all or just a portion of the analysis of the EEG signal samples isperformed in an EEG detection system (e.g., an ASIC integrated with orin communication with EEG sensors).

FIG. 3 is a functional diagram illustrating an EEG detection system inaccordance with some embodiments. As shown, the EEG detection system 130includes a processor 132 (e.g., an FPGA or ASIC), active EEG sensor 136,and a reference EEG sensor 138, and a communication link 134. Themeasured EEG signals are provided to the EEG control system 110. In someembodiments, a continuous measure of EEG signal samples are detected andprovided to the EEG control system 110.

FIG. 4 illustrates an EEG detection system including an EEG sensor andreference EEG sensor mounted inside a hat in accordance with someembodiments. As shown, the EEG detection system 130 is a hat to be wornby the user that includes the EEG sensor 136 and the reference EEGsensor 138 inside the hat. The EEG sensor 136 and the reference EEGsensor 138 are connected via a wired line communication (e.g., serialcommunication link) to the EEG control system 110. In some embodiments,the EEG sensor 136 is located inside the hat to be on the occipitalregion of the user's head (e.g., for visual event related EEG signaldetection) when the hat is worn by the user, and the reference EEGsensor 138 is located in another location on the user's head (e.g., onthe forehead, a side of the user's head above an ear, or on the back ofthe user's head in a location different than the location of the activeEEG sensor). In some embodiments, the EEG sensor(s) are located indifferent locations based on the type of stimulus events to be detectedas will be appreciated by those of ordinary skill in the art. In someembodiments, the EEG sensor 136 and reference EEG sensor 138 arenon-contact EEG sensors. In some embodiments, the EEG detection systemincludes more than one EEG sensor 136. In some embodiments, the EEGdetection system 130 is in the form of a headset, audio headset, anautomobile seat headrest, or any other form of apparatus or module thatcan be used by the user to securely locate the EEG sensor(s) 136 and thereference EEG sensor 138 on appropriate locations of the user's head forEEG signal detection. In some embodiments, the EEG detection system 130(e.g., a hat/cap, as shown) includes a grounded ear clip to reduce theamount of noise.

FIGS. 5A-B illustrate EEG sensors in accordance with some embodiments.As shown in FIG. 5A, the EEG sensor 136 is mounted on the inside of theEEG signal detection hat. As shown in FIG. 5B, a top-side of the EEGsensor 136 is illustrated, in which the illustrated EEG sensor 136 is anon-contact electrode that is approximately the size of a U.S. quartercoin. The EEG sensor 136 is integrated into a printed circuit board(PCB) with analog front-end circuitry that amplifies the EEG signal andfilters out noise. The EEG sensor 136 includes a metal shield, whichprotects, for example, the sensitive signal from external noise. In someembodiments, the circuitry for the EEG sensor 136 is integrated into anASIC.

FIG. 6 is another block diagram illustrating an EEG system for SEPs inaccordance with some embodiments. As shown, the EEG system for SEPs 100includes a computer 610 configured (e.g., programmed) to perform an SEPdetermination algorithm for detected EEG signals, a controller 620 forcontrolling LED (flashing) lights system 650 (e.g., controlling thetiming of onsets and offsets of light flashes and which lights flash inwhich patterns), and EEG circuitry 630 for receiving (and in someembodiments, processing) detected EEG signals from the EEG sensor 136and the reference EEG sensor 138. As shown, four LED lights are providedin the LED lights system 650. In some embodiments, one or more LEDlights are provided.

The controller 620 also includes an FPGA 622 (or, in some embodiments,any other form of a processor or software executed on a processor, suchas an ASIC or programmed processor). In some embodiments, the controller620 controls the LED lights 650 and also communicates with the computer610 and the EEG circuitry 630. In some embodiments, the controller 620controls the flashing lights and receives EEG signal (sample) data fromthe EEG circuitry 630. In some embodiments, the controller also combinesthe received EEG signal data and light timing data (e.g., for theflashing onsets/offsets of the LED lights system 650) into a serialstream that is sent to the computer 610 for further analysis andprocessing (e.g., using a real-time SEP determination algorithm). Insome embodiments, the controller 620 also sends control signals to acontrolled device (e.g., the device 150).

The EEG circuitry 630 includes firmware 632 (or, in some embodiments,any other form of a processor or software executed on a processor, suchas an ASIC or FPGA or programmed processor). The controller is in serialcommunication with the computer 610 and the EEG circuitry 630, as shown.In some embodiments, the EEG circuitry 630 is also directly connectedto, as shown via a direct serial connection (or, in some embodiments, indirect communication, wired or wireless) with the computer 610. In someembodiments, one or more of these connections are wireless.

FIG. 7 illustrates an EEG detection system with non-contact EEG sensorsin accordance with some embodiments. As shown, the EEG detection system130 is in the form of a hat or cap worn by the user, which includes abattery 710 (e.g., a rechargeable lithium ion battery), EEG circuitry720, and wiring to the EEG sensors 730 (the non-contact EEG sensors aremounted inside of the hat and, thus, not visible in this depiction ofthe hat being worn by the user). In some embodiments, the EEG detectionsystem 130 is in wireless (e.g., Bluetooth or another wireless protocol)with other apparatus/devices, such as the EEG control system 110. Insome embodiments, the battery 710, EEG circuitry 720, and wiring to theEEG sensors 730 are more tightly integrated into the hat/cap and/orother head wear apparatus (as similarly discussed above), and, in someembodiments, generally not visible when worn by the user. As will beappreciated, various other designs and head wear apparatus can be usedto provide the EEG detection system 130 disclosed herein.

FIGS. 8A-B illustrate LED lights for an EEG system for SEPs inaccordance with some embodiments. As shown, FIG. 8A illustrates the LEDlights 650 in which all four LED lights are turned off (e.g., notflashed on). As shown, FIG. 8B illustrates the LED lights 650 in whichall four LED lights are turned on (e.g., flashed on). As shown, the LEDlights 650 are mounted in a box shaped apparatus, which, for example,can flash in patterns that represent commands to a user. In someembodiments, each of the four separate LED lights of the LED lightssystem 650 can flash on/off independently. In some embodiments, the LEDlights of the LED lights system 650 flash at distinct fixed frequencies.In some embodiments, the LED lights of the LED lights system 650 flashat variable frequencies in a fixed pattern. In some embodiments, each ofthe four LED lights flashes at a different frequency. In someembodiments, the frequencies of each of the LED lights are separated by1 or 2 Hz (e.g., the four LED lights can be set at the followingfrequencies: 9 Hz, 10 Hz, 11 Hz, and 12 Hz, or other frequencies, suchas in the range of 8 Hz to 20 Hz or some other frequency range for whichSEPs can effectively be detected). In some embodiments, fewer than fouror more than four LEDs are included in the LED lights system 650. Insome embodiments, the flashing pattern is controlled by the controller620 (e.g., controlling the frequency of the flashing using an FPGAcontroller, such as a Xilinx FPGA chip that executes Verilog code).

FIGS. 9A-B are charts illustrating EEG data and light control signaldata for an EEG system for SEPs in accordance with some embodiments.FIG. 9A illustrates measured EEG signals (in Volts (V)) relative to time(in seconds). FIG. 9B illustrates the light input signal (in Hertz (Hz))(e.g., flashing light events) relative to time (in seconds). As shown,the light input signal is a square wave of a fixed frequency.

FIG. 10 is a power spectrum chart for sample EEG data. In particular,FIG. 10 illustrates measured EEG signals (in V) relative to frequency(in Hz), in which there are two measurements depicted, a first measuredEEG signal for which no light events are present, and a second measuredsignal for which 12 Hz light events occurred and the user was looking ata light that flashed on and off at a fixed frequency of 12 Hz. Asillustrated in this example shown in FIG. 10 and as discussed above, itis difficult to determine if the increased power at 12 Hz is due tolooking at a 12 Hz light, or if it is irrelevant noise.

FIG. 11 is a chart illustrating averaged EEG data following light onsetsfor an EEG system for SEPs in accordance with some embodiments. Inparticular, FIG. 11 illustrates an averaged EEG signal following lightonsets (e.g., a 12 Hz flashing light event) in which the average of thetime series of data is time-locked to the 12 Hz flashing light. Asshown, the flash-locked average signal provides a signal shape that canbe recognized to detect that the light is being attended (e.g., observedby the user), such that an SEP is effectively detected (e.g., using athreshold comparison and/or a signature signal comparison, as discussedherein).

FIG. 12 is a chart illustrating a correlation of flash and raw EEG datafor an EEG system for SEPs in accordance with some embodiments. Inparticular, FIG. 12 illustrates a correlation analysis between averagedEEG signals (e.g., flash-locked average signals) and flashing lightevents (e.g., light flashes). At each discrete point in time, themeasured EEG signals and the measured light signals are multipliedtogether. All of these results are averaged together to form acorrelation number. In some embodiments, the analysis is repeated usingtime offsets between the EEG data and the light signal data (e.g., 30ms). The result is a characteristic shape that can be used to determinethat the light is being attended (e.g., observed by the user), such thatan SEP is effectively detected.

FIG. 13 is a flow chart for an EEG system for SEPs in accordance withsome embodiments. At 1302, the process begins. At 1304, multiple EEGsignal samples are detected. At 1306, a stimulus-locked average signalis generated using the EEG signal samples. In some embodiments, anaverage of the EEG signal is determined that is time-locked to the lightonset and/or offset event(s). For example, for a specified period (e.g.,50 ms) after a light onset the EEG signal is recorded, and suchrecording is performed after each of one or more light onsets. Theresulting 50 ms EEG segments are then averaged together to provided theaveraging signal. At 1308, whether the EEG signal samples are evoked inresponse to a pattern of stimulus events (e.g., an SEP determination,such as in response to a visual event) is determined. For example, acharacteristic shape of the stimulus-locked average signal can bedetected using various techniques including comparing to a threshold atsome delay from the onset, integrating the averaged EEG signal over sometime period and comparing that result to a threshold, or creating aclassifier that distinguishes if the light is being attended. As anotherexample, a prototype of an ideal average (e.g., when a light is beingattended) can be constructed and multiplied by the actual average. Ahigh value result will indicate an attended light. The prototype can beconstructed in a variety of ways, including computing EEG averages whena user is known to be looking at the light, or constructing anauto-regression model of EEG data when a user is known to be looking atthe light, and using the coefficients of that auto-regression model asthe data elements of the prototype. At 1310, a control signal isprovided based on the SEP determination. At 1312, the process iscompleted.

FIG. 14 is another flow chart for an EEG system for SEPs in accordancewith some embodiments. At 1402, the process begins. At 1404, multipleEEG signal samples are detected and recorded (e.g., stored). At 1406, astimulus-locked average signal is generated using the EEG signalsamples. At 1408, a peak value for the stimulus-locked average signal iscalculated, in which the first peak value is determined based on maximumvalue minus a minimum value of the averaged signal. At 1410, the peakvalue is compared with a threshold value. For example, if the userexperienced the stimulus evoked event (e.g., was looking at the flashinglight), then the averaged EEG signal will generally include a noticeablepeak shortly after the flash onsets (e.g., after an offset of about 30ms to 50 ms for such SEPs). As a result, a threshold (e.g., signaturesignal) can be set up to determine if a peak exists. In someembodiments, the system is trained based on testing with a particularuser, and the threshold values (or, in some embodiments, signalsignatures) are generated based on the training. In some embodiments,the EEG signal samples are correlated with the pattern of stimulusevents using a time delay offset (e.g., 30 ms to 50 ms) to determinewhether the EEG signal samples are evoked in response to a pattern ofstimulus events (e.g., an SEP determination, such as in response to avisual event). At 1412, a control signal is provided based on the SEPdetermination. At 1414, the process is completed.

FIG. 15 is a diagram illustrating different stimulus frequency types inaccordance with some embodiments. As shown, a stimulus frequency for thelight input signal can be a single, fixed frequency, such as a squarewave, a sign or triangle wave, or a carrier with modulation. In someembodiments, a mixed frequency stimulus is used, in which a mixedfrequency stimulus is, for example, a combination of two or more fixedfrequencies added together. In some embodiments, various other types ofnon-periodic signals are used and are matched with time domain analysis.For example, a carrier wave with modulation would be similar to an FMradio signal with a large sine-wave component at a fixed frequencycombined with a different smaller signal. As another example, asingle-frequency stimulus can be adjusted by adding some variation inthe frequency. As will be appreciated, there are also many ways toconstruct pseudo-random codes, such as CDMA codes used in cell phonenetworks.

FIG. 16 is a diagram illustrating time domain algorithms in accordancewith some embodiments. In some embodiments, time domain classifiertechniques use the EEG signal without a conversion to the frequencydomain. For example, one approach that can be used is to multiply theEEG signal by sine waves of the same frequency as the flashing light. Ifthe sine wave has the correct frequency and phase, the output will havea relatively large amplitude. The absolute value of such an output willbe relatively high, compared with incorrect sine waves. As anotherexample, a correlation analysis can be used that would multiply eachpoint of the EEG data with each point of the light intensity data (e.g.,after the means are subtracted away). If there is no correlation betweenthe two vectors of data, the output will be zero. The correlation canalso be computed between the light intensity data and a delayed versionof the EEG (e.g., using an appropriate offset, such as 30 ms to 50 ms).At some delay, the correlation will typically be strong for a light thatis being attended. Various embodiments of stimulus-locked averagingtechniques are further described below.

FIG. 17 is a chart illustrating an example of four stimulus-lockedaverages in accordance with some embodiments. One of the averages hasdeveloped a characteristic shape, and the other averages are relativelyflat.

FIG. 18 is a chart illustrating an example for generating a flash-lockedaverage signal in accordance with some embodiments. As shown in FIG. 18,an EEG signal from a user looking at a flashing light and a signal thatis used to control the light are both measured. Segments of the EEGsignal of a fixed length (e.g., 100 ms) that follow the light onsets arefirst recorded. The recorded data is then averaged together such thatthe first EEG data points following light onsets are averaged together,then all the second data points that follow light onsets are averagedtogether, then the third data points that follow light onsets areaveraged together, and so forth. The result provides an average waveformof the same length as the recorded segments. For example, if the userlooked at the light, then the averaged waveform will include acharacteristic shape (e.g., a positive deflection in EEG voltage about30 ms to 50 ms after the light onset time). This characteristic shapecan be detected in a variety of ways. If the user did not look at thelight the averaged waveform will be mostly flat. In some embodiments,this technique is similarly used for generating a stimulus-lockedaverage signal. In some embodiments, the measured EEG signals includeinvoluntary EEG signal responses. In some embodiments, the measured EEGsignals also include voluntary EEG signal responses, such as intensityor focus related EEG signal responses (e.g., the strength of the EEGsignal can be altered by the way in which (such as a periphery versus adirect focus) or the intensity with which a user looks at a flashinglight(s)).

Although the foregoing embodiments have been described in some detailfor purposes of clarity of understanding, the invention is not limitedto the details provided. There are many alternative ways of implementingthe invention. The disclosed embodiments are illustrative and notrestrictive.

1. A system, comprising: a processor configured to: generate astimulus-locked average signal of a plurality of EEG signal samples; anddetermine whether the plurality of EEG signal samples are evoked inresponse to a pattern of stimulus events; and a memory coupled to theprocessor and configured to provide the processor with instructions. 2.The system recited in claim 1, wherein the pattern of stimulus eventsincludes a plurality of light sources emitting light flashesperiodically at a fixed frequency.
 3. The system recited in claim 1,wherein the pattern of stimulus events includes a plurality of lightsources emitting light flashes periodically, including a first lightsource emitting light flashes a first fixed frequency and a second lightsource emitting light flashes at a second fixed frequency.
 4. The systemrecited in claim 1, wherein the pattern of stimulus events includes alight source emitting light flashes periodically at a plurality offrequencies, including a first light flash at a first frequency and asecond light flash at a second frequency.
 5. The system recited in claim1, wherein the pattern of stimulus events includes a plurality of lightsources emitting light flashes at non-periodic frequencies.
 6. Thesystem recited in claim 1, wherein the pattern of stimulus eventsincludes one or more of the following: a pattern of visual stimulusevents, a pattern of audio stimulus events, and a pattern of tactilestimulus events.
 7. The system recited in claim 1, wherein the patternof stimulus events includes a plurality of light sources that eachprovides independent flashing light events.
 8. The system recited inclaim 1, wherein the plurality of EEG signal samples are evoked as aninvoluntary response of a user of the system.
 9. The system recited inclaim 1, wherein the plurality of EEG signal signals include voluntaryEEG signal responses.
 10. The system recited in claim 1, wherein each ofthe plurality of EEG signal samples includes a plurality of samplepoints.
 11. The system recited in claim 1, wherein each of the pluralityof EEG signal samples is a time series of EEG signal samples.
 12. Thesystem recited in claim 1, wherein the stimulus-locked average signalincludes a flash-locked average signal.
 13. The system recited in claim1, wherein the determination of whether the plurality of EEG signalsamples are evoked in response to a pattern of stimulus events isperformed by comparing the stimulus-locked average to a threshold. 14.The system recited in claim 1, wherein the determination of whether theplurality of EEG signal samples are evoked in response to a pattern ofstimulus events is performed by multiplying the stimulus-locked averageby a previously created prototype average.
 15. The system recited inclaim 1, wherein the determination of whether the plurality of EEGsignal samples are evoked in response to a pattern of stimulus events isperformed by multiplying the stimulus-locked average by a prototypeaverage that adapts to the user's EEG signals.
 16. The system recited inclaim 1, wherein the determination of whether the plurality of EEGsignal samples are evoked in response to a pattern of stimulus events isperformed by integrating the EEG signal data to generate a result, andcomparing the result to a threshold.
 17. The system recited in claim 1,wherein the determination of whether the plurality of EEG signal samplesare evoked in response to a pattern of stimulus events is performedusing a time domain classifier.
 18. The system recited in claim 1,wherein the determination of whether the plurality of EEG signal samplesare evoked in response to a pattern of stimulus events is performed inreal-time.
 19. The system recited in claim 1, wherein the processor isfurther configured to: determine whether a user of the system is lookingat a first light source.
 20. The system recited in claim 1, wherein theprocessor is further configured to: calculate a first peak value of thestimulus-locked average signal, wherein the first peak value isdetermined based on maximum value minus a minimum value of thestimulus-locked average signal; and compare the first peak value with athreshold.
 21. The system recited in claim 1, wherein the processor isfurther configured to: correlate the plurality of EEG signal sampleswith the pattern of stimulus events using a time offset based onstimulus timing.
 22. The system recited in claim 1, wherein theprocessor is further configured to: train based on a first user todetermine a first signature signal to assist in correlating theplurality of EEG signal samples with the pattern of stimulus events forthe first user.
 23. The system recited in claim 1, further comprising: aplurality of electrodes including an active EEG electrode and areference EEG electrode, wherein the plurality of electrodes detect theplurality of EEG signal samples.
 24. The system recited in claim 1,wherein the processor is further configured to: provide a control signalbased on a determination of whether the plurality of EEG signal samplesare evoked in response to the pattern of stimulus events.
 25. The systemrecited in claim 1, wherein the processor is further configured to:provide a control signal to a device based on a determination of whetherthe plurality of EEG signal samples are evoked in response to thepattern of stimulus events, wherein the device includes one or more ofthe following: an entertainment system, an educational system, a medicalsystem, an automobile system, and a computer executing an application.26. A method, comprising: measure a plurality of EEG signal samplesusing a bio-signal sensor; generate a stimulus-locked average signalusing the plurality of EEG signal samples; and determine whether theplurality of EEG signal samples are evoked in response to a stimulus.27. A computer program product, the computer program product beingembodied in a computer readable storage medium and comprising computerinstructions for: record a plurality of EEG signal samples; generate astimulus-locked average signal using the plurality of EEG signalsamples; and determine whether the plurality of EEG signal samples areevoked in response to a stimulus.